Configure Snowpark Connection Details

Snowpark is the developer framework of Snowflake that lets you write code in the language of your choice and run it within Snowflake. This simplifies the architecture resulting in faster data processing, reduced latency, and improved data security. Instead of pushing data to external systems, you write your code in different languages like Python or Java and build pipelines and workflows leveraging Snowflake's scalability, security, and governance.

The Calibo Accelerate Platform supports ingestion and transformation using Snowpark to ingest data into a Snowflake data lake. After you save the connection details for Snowpark, you can use it in any of the nodes in a data pipeline.

Prerequisites

Ensure that the following requirements are met:

  • A Snowflake account with the required roles and permissions.

  • Outbound access to Snowflake endpoints.

  • Following privileges warehouse, database, schema:

    • USAGE

    • CREATE STAGE

    • CREATE PROCEDURE

Configuring Snowpark connection details

  1. Sign in to the Calibo Accelerate platform and click Configuration in the left navigation pane.
  2. On the Platform Setup screen, on the Cloud Platform, Tools & Technologies tile, click Configure.
  3. On the Cloud Platform, Tools & Technologies screen, in the Data Integration and Data Transformation section, click Configure.


    4. (After you save your first connection details in this section, you see the Modify button here.)
  4. On the Snowpark Integration screen, do the following:

    1. In the Details section, provide the following details:

      FieldDescription
      NameGive a unique name to your Snowpark configuration. This name is used to save and identify your specific Snowpark connection details within the Calibo Accelerate platform.
      DescriptionProvide a brief description that helps you identify the purpose or context of this Snowpark configuration.

    1. In the Configuration section, provide the following information:

      FieldDescription
      Snowflake Configuration

      Select the Snowflake account that you want to use for the Snowpark instance, from the dropdown menu. After you select the Snowflake account, you can view the following details:

      • warehouse

      • accountURL

      • databaseName

      • schemaName

      Note:

      Depending on the Snowflake account that you select when you configure Snowpark, the cloud provider, Orchestrator Agent, and Secret Manager configured for that account are pre-populated and are non-editable.

      Use Credentials from Snowflake Configuration

      In Calibo Accelerate, you can either use the same credentials as the Snowflake account or configure a separate Snowpark environment.

      • If the toggle is enabled, the same credentials as the Snowflake account are used for the Snowpark environment.

      • If the toggle is disabled, you can provide a different set of credentials for the Snowpark environment with restricted access:

        • Snowflake Account Admin User - Provide the Snowflake Admin username.

        • Snowflake Account Admin Password - Provide the Snowflake Admin password.

      Test ConnectionClick Test Connection to verify the provided details and ensure that the Calibo Accelerate platform is able to successfully connect to the configured Snowpark instance.
      Snowpark Package

      The current version of the Snowpark package used for the Snowpark environment is displayed.

  5. Secure configuration details with a password
    To password-protect your Snowpark connection details, turn on this toggle, enter a password, and then retype it to confirm. This is optional but recommended. When you share the connection details with multiple users, password protection helps you ensure authorized access to the connection details.

  6. Click Save Configuration. You can see the configuration listed on the Data Integration and Data Transformation screen.

Related Topics Link IconRecommended Topics

What's next? Templatized data ingestion using Snowpark